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Abstract 

A large number of different model checking approaches has been proposed during the last 
decade. The different approaches are applicable to different model types including untimed, 
timed, probabilistic and stochastic models. This paper presents a new framework for model 
checking techniques which includes some of the known approaches, but enlarges the class of 
models for which model checking can be applied to the general class of weighted automata. The 
approach allows an easy adaption of model checking to models which have not been considered 
yet for this purpose. Examples for those new model types for which model checking can be 
applied are max/plus or min/plus automata which are well established models to describe dif- 
ferent forms of dynamic systems and optimization problems. In this context, model checking 
can be used to verify temporal or quantitative properties of a system. The paper first presents 
briefly our class of weighted automata, as a very general model type. Then Valued Computa- 
tional Tree Logic (CTL$) is introduced as a natural extension of the well known branching time 
logic CTL. Afterwards, algorithms to check a weighted automaton according to a CTL$ formula 
are presented. As a last result, a bisimulation is presented for weighted automata and for CTL$. 

Key words: Finite Automata, Semirings, Model Checking, Valued Computational Tree Logic, 
Bisimulation. 

Subject Classification: D.2.4, F.3.1 

1 Introduction 

Model checking of finite state systems is an established approach for the automatic or semi- 
automatic analysis of dynamic systems from different application areas. The basic model checking 
approaches have been proposed for untimed models and allow one to check the functional correct- 
ness of systems. The general idea of this kind of model checking is to determine the set of states of 
a finite state automaton which satisfies a formula of a temporal logic. Common examples of modal 
logics to express formulas are Linear Time Logic (LTL) or Computational Tree Logic (CTL). For 
both logics, efficient analysis algorithms exist that allow the handling of extremely large automata. 
Nowadays, several software tools are available that include model checking algorithms, allow the 
automatic analysis of dynamic systems and have been applied to practical examples from different 
application areas like hardware verification or software engineering. An enormous number of papers 
on model checking and related topics exists, for relatively recent surveys we refer to jl3l I15| and 
|14j as a textbook. 

For several application areas, the proof of functional correctness is not sufficient to assure the 
correct behavior of a system. For instance, in real-time systems, it has to be assured that a function 
of a reactive system performs correctly and takes place in a given time interval. For other systems, 
we may tolerate some erroneous behavior if it occurs only with a sufficiently small probability. 
In this and similar situations, a basic proof of correctness is not sufficient. Consequently, model 
checking approaches have been extended to handle also timed, probabilistic and stochastic systems. 
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In |2j|, an extended version of the temporal logic CTL is presented that is denoted as Probabilistic 
Real Time Computational Tree Logic (PCTL). This logic allows the definition of properties which 
state that something will happen with a given probability in a fixed time interval. The logic is 
interpreted over finite Discrete Time Markov Chains (DTMCs). The timing is defined by the 
number of transitions that occur and probabilities are defined by the transition probabilities of the 
DTMC. PCTL is a useful logic to express requirements for real time systems with constant delays. 
Other model checking approaches analyze different forms of timed automata 1 that are possibly 
augmented by different timing models 0. 

The mentioned approaches for model checking are all similar but differ in various details. In par- 
ticular, the different logics are all interpreted over an appropriate automata model. The automata 
models used in the mentioned approaches are untimed automata for standard model checking, prob- 
abilistic automata for timed and probabilistic model checking, stochastic automata for stochastic 
model checking and different forms of timed automata. By considering the wide area of finite 
state automata, one can notice that apart from these automata types other models have been pro- 
posed and applied successfully in different application areas. Examples are min/plus, max/plus, or 
min/max automata that have been used for the analysis of real time systems 0, communication 
system and discrete event systems [21 ED]- Furthermore, similar models have been applied for 
natural language processing |27] or image compression [2^. It is quite natural and for most of the 
mentioned applications also very useful to extend model checking approaches to all these types of 
automata. Since the class of weighted automata provides in some sense a superset of different au- 
tomata types, which includes different forms of probabilistic automata and also untimed automata, 
one may strive for a general framework of model checking which can be applied to a wide variety 
of different types of weighted automata without defining a new approach for each type. Such a 
framework is of theoretical interest to get a better understanding of modelchecking and to get a 
common ground for model checking in various application areas. From a methodological point of 
view, it gives direct access to model checking techniques for various types of automata that do 
not profit from these techniques yet. Finally, it supports tool development: in an object oriented 
setting, implementation of a specific model checker can inherit basic techniques from a more general 
class that implements techniques valid for the whole framework. 

Weighted automata \17\ I25j are a well known class of automata where transitions are labeled 
with labels from a finite alphabet and, additionally, receive weights or costs that are elements of 
some semiring. A key observation is that the algebraic structure of a semiring is sufficient to define 
modelchecking for weighted automata. The advantage is that by selecting appropriate semirings, 
one obtains different types of automata that include most of the above mentioned types. This 
general type of automata is suitable to define a bisimulation as we did in |S1 ^] . In [Hj , the process 
algebra GPA has been introduced for the specification of models in a compositional way such that 
the underlying semantic model is a weighted automaton in the case of a finite set of states. 

In this paper, we develop a model checking approach for weighted automata. The approach 
allows us to check formulas of the newly defined logic Valued Computational Tree Logic (CTL$) 
over a weighted automaton. Algorithms for model checking are developed and it will be shown 
that by an appropriate definition of the semiring used for the definition of transitions weights, 
we naturally define model checking approaches for different model types without developing new 
approaches in each case. The special cases include untimed, probabilistic, min/plus, max/plus, and 
min/max automata such that known model checking approaches are covered and new approaches 
are introduced in the case of min/plus, max/plus, and min/max automata. By the use of other 
semirings for transition weights, the proposed approach applies to a wide class of automata models. 
In so far, we develop some form of a generic approach for model checking that is applicable to other 
model classes and that includes algorithms to perform model checking. 

The structure of the paper is as follows. In the next section, we present the automata model 
that is considered in this paper. Afterwards, we define CTL$, a logic for automata with transition 
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weights that is an extension of the well known branching time logic CTL for untimed automata. 
The following section introduces algorithms to check a CTL$ formula according to an automaton 
with transition weights. We consider algorithms with explicit state representations for clarity at 
this point. A treatment by symbolic representations like multi terminal binary decision diagrams 
(MTBDDs) is feasible but not in the focus of this paper. In Section |SJ bisimulation is briefly defined 
for automata with transition weights and it is proved that bisimilar automata are indistinguishable 
under CTL$ formulas. Afterwards, in Sectional we present several examples of concrete realizations 
of weighted automata. The paper ends with the conclusions. 

2 Weighted Automata 

To present our general automata model, we first introduce semirings that are needed to define 
labels for transitions. Afterwards the automata model is defined. 

Definition 2.1 A semiring (K, + , ^ ,iD,I) is a set K with binary operations + and ^ defined 
on K such that the following axioms are satisfied: 

1. + , ^ are associative, 

2. + is commutative, 

3. right and left distributive laws hold for + and ^ , 

4- (D andl are the additive and multiplicative identities with iD t^I, 
5. /c^(D = (D^A;=(D holds for all G K. 

Semirings can show specific properties like idempotency, commutativity, or being ordered; prop- 
erties that we formally define as follows. 

Definition 2.2 A semiring is ordered with some transitive ordering <, if a < b or b < a for all 
a,bGK. 

An ordered semiring preserves the order if for all a,b,c £ K.' 

a <b =^a + c<b + c, a^ c < b'^ c and c^ a < c^ b. 

A semiring is commutative if multiplication is commutative. 
It is idempotent if addition is idempotent. 

It is closed if infinite addition is defined and behaves like finite addition. 

Furthermore, we define a < b \l a < b and a ^ b. The supremum sup(a, b) of a, 6 G K is a if 
a > b and b otherwise, the infimum inf(a,6) is a if a < 6 and b otherwise. To make the notation 
simpler, we use sometimes K for the whole semiring and ab is used for a^b. 

The well known Boolean semiring (B, V, A, 0, 1) is order preserving, commutative, idempotent, 
and closed whereas (R>o, +, — , 0, 1) is order preserving and commutative, but not idempotent 
and not closed. The semirings (R>o U {— oo}, max, +, — oo, 0) and (R>o U {oo}, min, +, oo, 0) are 
order preserving, idempotent, and commutative, but not closed. However, they are closed if oo or 
respectively — oo are added. 

Definition 2.3 A finite weighted automaton over semiring K and over a finite alphabet C is a A 
tuple A = (5, a, T, 13), where 

1. S = {0, . . . , n — 1} is the finite state space. 
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2. a : 5 — > K is the initial weight function, 

3. T: SxCxS^His the transition function, 
4- /3 : 5 ^ K is the final weight function. 

The transition function T computes a transition weight for each label and each pair of states. 
Independently of the used semiring, T{x, a,y) = (D implies that no a-labeled transition between 
state X and state y exists. However, ID is defined differently in different semirings. Observe that the 
definition assures that between two states at most one transition exists that is labeled with a fixed 
label a. For some automata models, initial and final weight functions are not needed and hence 
usually not defined. If this is the case, the functions may be substituted by constant I, which is the 
neutral element according to multiplication; this allows a uniform formal treatment. 



Example 2.1 We consider a simple model of a driving test to illustrate the concept, Fig. ^ gives 
an automaton with S{A, B, . . . ,H,L} and C = {I, d, /, e}. Let L be the initial state, so we define 
the initialization function a{s) =1 if s = L and IS) otherwise. When a student starts, he/she takes a 
couple of lessons, which are transitions with label I (lesson). After at least one lesson, the student 
may be confident to start the test (state A) and drive around with the examinator (actions with 
labels d for drive). While driving the examinator may decide to finish (actions with label f) and 
to assign the desired driver's license in state H (Hooray). Alternatively, the student may make 
some errors (actions with label e for error) that lead him/her to less hopeful situations D, E, or 
F where after some further driving the examinator will finally decide to finish ( actions with label 
f ) and to refuse the license. This yields state G. Hence, the poor student can only return to L 
and take some more lessons. State H is the desired result, so we define /3(s) =1 if s = H and (D 
otherwise. We will consider this example with different semirings and different transition functions 
(assignment of weights), e.g., the Boolean semiring (B,V,A,0, 1) is useful to ask for existence 
of paths that lead to a driver's license (state H), or whether all paths lead to this state, i.e., if 
success is guaranteed. For the Boolean semiring, function T is defined by assigning I = 1 to all 
arcs present in -Fig. Q ([0, 1], +, -,0, 1) is useful to achieve a probabilistic model, where actions are 
randomly selected and one may ask for the probability to succeed. If one asks for trouble, one can use 
(R>oU{— cxd}, max, +, — oo, 0) to look for the hardest path to success, and (R>oU{oo}, min, +, oo, 0) 
for the one with minimal stress, given that weights indicate how much energy is necessary to perform 
that action. 

The class of weighted automata is known for a long time in automata theory ^7j- The concrete 
realization defined here has been proposed in 0. In a process algebra is presented that is 




Figure 1: Example automaton, a driving test model. 
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based on the above concepts in the sense that its dynamic behavior yields an automaton with 
transition weights; however a term in a process algebra may impose an automaton with an infinite 
number of states. Different semirings yield completely different automata. Before we present some 
concrete realizations by fixing the semiring, we consider general methods to analyze the behavior of 
automata with transition weights. The behavior of a weighted automaton considers the weights of 
paths between states where a path is described by a finite or infinite sequence of transition labels. 

To analyze the behavior of an automaton over all paths, we present an approach that is based on 
vector-matrix computations because this is a convenient approach to compute these results. Since 
we consider automata over finite state spaces and finite sets of transition labels, each automaton 
can be described by sets of K"^" matrices and K" vectors. Thus, we define for each a € £ a 
matrix Ma with Ma(x,y) = T{x,a,y) and M = J^aeC^a as a matrix that collects all weights 
independently of the labels. Furthermore, we define a row vector a with a(x) = a{x) and a column 
vector b with b(.x) = f3{x). To complete the notation let, I be the n x n identity matrix over 
semiring K, let ej S K" be a row vector with I in position i (0 < i < n) and (D elsewhere and let e-^ 
be a column vector where all elements equal to I. It is straightforward to define matrix sum and 
product using the operations of the semiring instead of the usual multiplication and addition. 

We start with the analysis of paths and introduce some notations first. We use x, y, z for 
states and i, j, k for running indices in sums or products. A path of automaton A is defined as 
a sequence of states and transitions starting in a state x G iS with a(x) 7^ D. In automata theory 
paths may be defined by sequences of states or transitions or both. We use here a definition that 
observes transitions via their labels and states. However, the approach can be easily restricted to 
observe only states or only transitions. Let vr be a path, vr^ the sequence of states in the path and 
TT* the transition labels. We denote by 7r| G iS (i = 0, 1, 2, . . .) the i-th state in the path and by 
TT* G Act {j = 1,2,...) the j-th transition label. Thus, tt = (7ro7r*7rf . . .) is a path of automaton A 
if T(7r|, vr*^-^, TTf_^i) 7^ iD. A path might be of infinite or finite length. In the finite case, index i runs 
from to jvrl where |7r| is the length of the path, i.e., the largest index i in the path. Let a be the 
set of paths of automaton A, a" (c-") the set of paths of length n (< n) and a!^ (<7^") the set of 
paths of length n (< n) that start in state x. For each finite path, we can compute the weights as 
(ce = costs each) 

ce(7r) =a(7r^)^n!l^«'^m'<+i)^/?(^K|) (1) 

where ni=i0^i = c^i ' • • • ' and the case of finite might be extended to A = 00, if the semiring 
is appropriately chosen such that the infinite product can be computed. 

If we focus on observing the behavior of an automaton by considering a sequence of labels 
seq = ai, . . . , am with Ui € C for a path vr, then one does not want to distinguish among paths tt 
and tt' that produce the same sequence seq. The weights summed over all paths with labeling seq 
starting in state x are given by (ca = costs all) 

m 

ca^ {seq) = a(x) ^ (H^^iMa, ) ^ b , (2) 

and the weights of all paths of length m with an arbitrary labeling is computed as 

ca^(*™) = a(x)^M™^b^ . (3) 

The above computation of weights assumes that a specific initial state is known. Alternatively, one 
can consider the case that vector a defines the weights of initial states. The weights of paths are 
defined then as 

--~-- m 

ca{seq) = a ^ (n,=i^«i) ^ ^ and ca(*"') = a ^ M"' ^ b. (4) 
Apart from the weights of paths, we consider possible terminating states and the weights of reaching 
those states. These values are described by a row vector 

d,e, = a-(n.^^M„J, (5) 
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such that ca{seq) = dgeq ^ b. 

3 Valued Computational Tree Logic 

The usual way of describing dynamic properties of a system are temporal logics which exist in 
various forms. Very popular is the branching time logic CTL ^2j. CTL formulas are interpreted 
over labeled transition systems and efficient algorithms for model checking finite systems exist 
and have been implemented in software tools _^T6!. CTL allows us to check properties of paths 
of an automaton where an all- or existence-quantifier has to precede any path quantifier. Since 
CTL is defined for transition systems where transitions are not quantified, it cannot be used to 
derive properties that hold with a certain probability or hold for a specified time. To express such 
probabilities, the logic has to be extended as done by several authors. The logic RTCTL is described 
in as an extension of CTL. RTCTL, in contrast to CTL, allows reasoning about times. Thus, 
it can be expressed that a property will become true within 50 time units or that a property holds 
for 20 time units. Time is discrete in this model and one transition takes exactly one time step. In 
|21j . the logic PCTL is introduced that can be used to describe properties that hold for some time 
(or after some time) and hold with at least a given probability. Thus, this logic extends RTCTL 
with respect to probabilities. Formulas of PCTL are interpreted over discrete time Markov chains 
(DTMCs) and the model checking problem for PCTL is polytime decidable jSj. In this model, time 
is also discrete and one transition lasts one time step. 

In this paper, we extend CTL by defining a logic for weighted automata. This approach is 
more general than the previous extensions of CTL because it can be applied to a large number of 
models by defining an appropriate semiring structure for quantifying transition labels. Since our 
automata model contains transition labels we extend our logic by propositions that allow us to 
reason over labeled transitions as it is done in Hennesy-Milner logic [221l26j . In this respect. Valued 
Computational Tree Logic (CTL$) might not be the natural name for the logic. However, since 
CTL is included in the logic CTL$ as a special case of automata over the Boolean semiring, we 
choose this name. We will show later that the approach includes probabilistic systems, although 
the presented logic is in these cases not completely equivalent to the different logics proposed for 
the models mentioned above. We will come back to this point in Section El where we present 
concrete realizations of our model. Here, we first define basic CTL$ formulas, introduce informally 
the semantics of a formula, and define some derived expressions afterwards. 

Definition 3.1 For a given set of atomic propositions, the syntax of a CTL$ formula for a semiring 
K is defined inductively as follows: 

• An atomic state proposition ^ is a CTL$ formula, 

• if <I>i and ^2 o,re CTL$ formulas, then ^<I>i and <I>i V ^2 oltc CTL$ formulas, 

• if ^ is a CTL$ formula and p G K, then [a]c^p.^ is a CTL$ formula, and 

• if^i and ^2 o'^e CTL$ formulas, t is a nonnegative integer or 00 andp G K, then <I>i ^2 
and <I>i AU^^p ^2 CTL$ formulas 

where coG {<, <, =, >, >}. 

Formulas of CTL$ are interpreted over weighted automata. A necessary condition to interpret 
a formula for an automaton is that both use the same semiring K, which will be assumed in the 
sequel. Atomic propositions of the kind $ : 5 — s- B describe elementary properties that hold or 
do not hold in a state s G 5 of an automaton. The goal of model checking is to compute the set 
of states for which a CTL$ formula ^ holds. Before we define formally for which states a formula 
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holds, we present the intuitive meaning of the formulas, i.e., we describe under which conditions 
formula $ holds for state x. 

• An atomic proposition $ is true in x e S, if the proposition holds in x. 

• -1$ is true in x if $ is false in x; $i V $2 is true in x if $1 or $2 are true in x. 

• [ctjxip-*^ is true in x if w txi p holds where w denotes the sum of weights of a-labeled transitions 
that leave x and end in some state where $ holds. 

• $1 U^p $2 is true in x, if w i^i p holds where w denotes the total amount of weights for 
all paths that 1) start in x and 2) fulfill $1 until they reach a state where $2 holds, and 3) 
perform at most t steps for condition 2). This operator ignores those paths that fail on any 
of the conditions 1) - 3). 

• $1 AU^p ^2 is true in x, if all paths that 1) start in x, 2) fulfill $1 until they reach a state 
where $2 holds, 3) require for this at most t steps and for the sum of the weights w of all 
these paths w >n p holds. This operator is more strict than the previous one, it requires all 
paths to observe conditions 1) - 3). 

We use the notations x \= ^ if x satisfies formula $ and -ix |= $ if this is not the case. The 
meaning of the first two cases above is obvious. For a formal definition of the last three cases, we 
make use of a description by vectors and matrices and introduce some additional notations first. 
Let for some matrix R G K"^" and two CTL$ formulas $1 and $2, R-I^i, ^2] £ K"^" be defined 



Consequently, matrix M[$i,$2] (Ma[$i,$2]) contains all transitions (labeled with a e C) that 
start in a state where $1 holds and end in a state where $2 holds and ![<!>, is a matrix that contains 
I in the main diagonal whenever $ holds for the corresponding state and all other elements are ID. 
Furthermore, let for some vector x, x[$] = xl[$, $]. With these notations, we can formally define 
the meaning of the presented CTL$-formulas using vectors and matrices rather than considering 
specific paths. 

• X \= [a]txip.$ if and only if w tx\p with w = ea;Mae^[<I>]. 

• X 1= $1 U^p $2 if and only if w txip with 



• X \= AU^p $2 if and only if a:: |= $1 U^jp $2 and for all tt G CTx exists some m < t such 
that TT^ 1= $2 A 7r| |= $1 A -i7r| |= $2 for < i < m. 

If a semiring is ordered, preserves the order by its operations and (D is the infimum of K, then 
a + 6 = (D implies a = 6 = (D. In that case, we can equivalently rewrite the condition on paths vr for 
X \= AUl^p $2 by requiring that the sum of weights of paths that contradict the property is ID. 
More formally. 



as 




w = 




1 A -•$2,*2]^b[$2] if t > 



ift = 



a(x) ^ ^ ((eLo(M[$i A -$2, $1 A -$2])'=) ^ M[$i A -$2, -^1 A -$2] 



(6) 



+ (M[$i A -.$2, ^1 A -'$2])*) ^ 
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for t>0. For i = 0, we have x |= $i C/j^p ^2 ^ x \= AUl^p $2- 

CTL$ contains an all but no existence quantifier. The reason for this decision is that the 
existence quantifier can often be described by the general path quantifier U using which 
indicates for many, but not for all semirings that a path of length < t exists that observes the 
required properties. For instance, the boolean semiring is a case where f7^,j) is suitable to decide 
existence of a path. 

Another reason for not introducing an existence quantifier for paths is that in general semirings 
this quantifier is not indistinguishable under bisimulation. Thus, bisimilar automata (see Sect. 
still might be distinguished via CTL$ formulas including path quantifiers considering single paths 
and this is in some sense against the idea of bisimulation and its connection to logics. Note that the 
quantifier AU does not introduce problems for order preserving semirings where D is the infimum 
and these semirings will be considered in the algorithms presented below. AU is necessary to make 
CTL$ equivalent to CTL if weighted automata are defined over the Boolean semiring. Since CTL$ 
shall not be less expressive than CTL, AU must be included. 

Several other operators can be derived from the basic operators of CTL$ . The basic operators 
A and are derived in the obvious way. By help of negation, one can show that for path formulas 
with $1 U^p <1>2, not all operators for comparisons cxi are essential. We present the relation for 

f/[^p <1>2 and omit index p, for readability. 

^1 C/^ «>2 = - ((^-i Ui<l>2) V ($1 Ui<^2)) f/< ^'2 = -(^1 C/>^2) 

$1 Ul ^2 = (^1 UL<^2) V ($1 Ul^2) ^1 UL ^2 = ($1 Ui<^2) A -(«>i Ul^2) 

The last equality shows that we may as well use ixi€ {>,>} to derive all other relations for 
$1 Ul^p $2- This will be done in the following section because > and > can be easily checked in 
the algorithms. 

Similarly, we have the following relation for [ 

'^Imp where p is again omitted for readability. 

[a]< = -■ {[a]= V [a]>) , [a]< = ^[a]>, [a]> = [a\= V [a]> and [a]= = [a]> A -'[a]> 

Again, it is sufficient to consider ixiE {>,>}. 

The following abbreviations are defined by extending the corresponding CTL$ formulas. 

• AX^p <I> = true AU^^^^ and UX^p^ = true U^p^ 

• AF^ $ = true AU^^p^ and UF^p^ = true U^^p^. 

X corresponds to a next operator. F denotes a finally operator. Such operators are common 
syntactical sugar of modal logics. 



4 Model Checking CTL$ Formulas 

To perform model checking in an efficient way, we restrict the semiring used for transition valuation. 
We assume that the semiring is ordered, that the order is preserved by the operations, and that (D is 
the infimum of K. Observe that these conditions are satisfied in most practically relevant semirings, 
e.g., in the examples presented below. To illustrate the point, (R, +,-,0, 1) is not ordered due to 
the fact that a < b does not imply a-c<6-c if c<0, but (R>o, +, -,0, 1) is ordered. This means 
that we prohibit negative weights, which is a common and familiar restriction in consideration of 
automata with transition weights. At the end of the section, we briefly outline when and how 
model checking can be performed for more general semirings. 

We follow other model checking approaches like |12) and define inductively over the length of a 
formula how a formula is checked. 
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• leng{^) = 1 if $ is an atomic proposition, 

• leng{-'^) = leng{^) + 1, 

• leng{^i V $2) = m.ax{leng{^i), leng{^2)) + 1, 

• leng{[a\^p.(^) = leng{^) + 1 and 

• leng{^i U^p $2) = leng{^i AU^^ $2) = Taax{leng{^i),leng{^2)) + 1- 

As in CTL model checking the set of states satisfying a formula of length I is computed after all 
sets of states that satisfy sub-formulas of length < I are known. Computation of the sets of states 
that observe atomic propositions, or $1 V $2 is identical to the corresponding computations in 
CTL. Thus, the new cases are [a]oc]p.$, $1 $2, and $1 AU^p $2- We describe a procedure for 
each of the three formulas that computes for each state whether it observes the formula or not. We 
present only cases of ixiG {>,>}, since the other cases can be derived from these cases as shown 
above. Let marked(x) be a variable that is true if a; ^ <1> and false otherwise. For the presentation 
of the algorithms, we use the vector matrix representation of the automaton which is also well 
suited for an implementation of the algorithms. 

An algorithm to compute [ajtxp.^. 

for (all X G <S) do 
marked (x) := false ; 
sum := D ; 

for (all y with Ma{x,y) ^ ID) do 
if (y 1= $) then 

sum := sum + Mo(a;, y) ; 
if (sum ixi p) 

marked (x) := true ; 

break ; 

The inner for-loop can be left if sum ixi p because due to our assumptions the value of sum 
cannot be reduced according to < or <. 

An algorithm to compute $1 U^^p $2 for t < 00. 

1. for (all z € 5) do 

2. if (x ^ $2) then 

3. w(.x) := b(.x) ; 

4. if (a(x) ^ b(a;) cxi p) then 

5. marked (x) := true ; 



6. 



else 



7. 



marked (x) := false ; 



8, 



else 



10. 



9. 



w(x) := iD; 

if (-1X 1= $1) then 



11. 



marked(x) := false ; 
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12. 



else 



13. marked(a;) := undefined ; 

14. V := M[$i A -i$2, $2] ^ w ; 

15. u := V ; 

16. for (all X E S with marked(x) = undefined) do 

17. if (a(a;) ^ u{x) xi p) then 

18. marked(a;) := true ; 

19. / := 2; 

20. while {I < t and 3a: € 5 with markcd(x)=undefined) do 

21. w := M[$i A -.$2,^*1 A -.$2] ; 

22. u := u + w ; 

23. for (all x G S with marked(a;) = undefined) do 

24. if {a.{x) ^ u(x) co p) then 

25. marked(a:) = true ; 

26. / := / + 1 ; 

27. V := w ; 

28. for (all x E S with marked(x) = undefined) do 

29. marked(a;) := false ; 

Steps 1 through 13 of the algorithm describe the initialization phase, several special cases are 
decided directly. A state x that satisfies $2 also satisfies 11^^ $2 if a(a^) ^ b(a;) ixi p. If 'W = 
">" and -i(a(a;) ^ b(a;) > p) then x does not satisfy the formula, because on all paths starting in 
X $2 immediately holds for the first time and the weights of these paths are too small such that 
the whole formula is false. For states where $1 and $2 both do not hold, the formula is false too. 
In the remaining cases, it is not clear yet whether the formula holds or not and those states are 
marked as undefined with respect to this formula. Steps 14 through 18 describe the first transition 
going from a state where $1, but not $2 holds into a state where $2 holds and check whether 
the formula becomes true by paths of length one. In the steps 19 through 27, transitions of the 
automaton between states where $1 but not $2 holds are mimicked step by step. In each step /, 
we compute per state x, where only <I>i holds, the sum of weights of paths of length / that end 
in a state where $2 holds and that pass through states where only $1 holds. These weights are 
collected in vector w. The weights of all those paths of length of at most I are accumulated in 
vector u. The iteration stops if t steps have been computed, which means that all paths of length 
< t have been considered, or if all states are classified, i.e., no state is marked as undefined. After 
leaving the iteration over all paths of length < t, all states that are still marked undefined do not 
satisfy the formula because no appropriate path can be found for them. The procedure eventually 
stops for finite t. 

For t = 00, the situation is different. In principle, we can use the above procedure, but it 
cannot be assured whether it stops or yields to an infinite computation. The crucial point is the 
computation of the infinite sum of matrices 

— ~ oo 
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The following relation holds if M[$i A ^^2, A -■$2] can be reordered to an upper triangular 
matrix. 

- — ~ oo - — ~ n 

^^^^(M[$i A -«>2, $1 A -$2])'' = 5I;.^o(M[$i A -^>2, ^1 A -^>2])^ 

The relation is true since R*' = for A; > n if R € K"'"' is an upper triangular matrix. In this case 
and for t > n 

such that the above algorithm for finite t can be applied for the infinite well. 

For the general case where M[<I>i A -'<I'27 'I'l A -'<1*2] cannot be reordered to an upper triangular 
form, computation of N[$i A -■$2; ^1 A -'<I>2] requires that the semiring K is closed and the concrete 
computation depends on the used semiring. We will give some examples for different semirings 
below. If N[$i A -i<I>2,<I>i A -■^'2] is available, then $1 U^p ^2 can be checked using an extension 
of the algorithm for the finite case, where the steps 19 through 29 are substituted by the following 
steps. 

u := N[^i A -■<I>2, ^1 A -■<I>2] ^ V ; 

for (all X € 5 with marked(x) = undefined) do 

if (a(a;) ^ u(x) cxi p) then 
marked (x) := true ; 

else 

marked (x) := false ; 
An algorithm to compute <I>i AU^p ^2- 

To analyze x |= <I>i AU^^p ^2, first x \= U^p ^2 has to be proved with the presented algorithm 
and then has to be checked. Since we restrict ourselves to an order preserving semiring K with 
ID as its infimum, the following result holds for R e K"'", a, b E K". 

-■ — - 00 - — ~ n 

aV R'^b >0 ^ aV R'-b > 

The result holds since the existence of a path between two states implies the existence of a path of 
length < n between these states (remember that |iS| = n). Thus @ becomes 

a(x) • • f (Efc=o (Mi^-i A -^>2, $1 A -^>2])'') • M[$i A -$2, A -$2] 

+ (M[$i A ^$2, ^1 A ^^a])""'"^*'"^) ^ e'^ = iD . 

This relation is checked for t > in the following algorithm where we assume that marked(x) is 
true if X 1= <I>i U^p $2 and false otherwise. 

1. w := M[<I>i A -■<I>2, A ^<1>2] ^ ; 

2. for (all X G 5 with marked (x) = true and w(x) > (D) do 

3. marked(x) = false ; 

4. A: := 1 ; 

5. while {k < min(t, n) and 3x with marked(x) = true) do 

6. w := M[<l>i A -■<I>2, 'I'l A ^$2] ^ w ; 

7. for (all X € 5 with marked(x) = true and w(x) > ID) do 

8. marked (x) = false ; 
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9. k:=k + l; 

10. w := e^[^>i A ^^>2] ; 

11. for (A; = 1 to min(t, n)) do 

12. w := M[<I)i A -■<I>2, 'I'l A -■$2] ^ w ; 

13. for (all 2; G 5 with marked(x) = true and w(x) > 0) do 

14. marked(3;) = false ; 

The procedure checks both conditions on which $1 AU^p ^2 may fail separately and requires a 

finite effort due to the finite summations. ^ 

Evaluation of $1 U^p ^2 and <I>i AU^jp $2 involves computation of R* and Y^k=o^'' sub- 
problems. In the algorithms given so far, those subproblems are solved by successive matrix-vector 
multiplications, which avoids an explicit computation of P = R and Q = J2k=o^ ■ space 
used to represent P or Q is tolerable for an application, those matrices can be computed with 
less steps by using iterated squaring if the semiring is idempotent. Iterated squaring is known 
for long, e.g., to compute a transitive closure of graph which corresponds to the boolean semir- 
ing. To compute P, we can use a binary representation of t = X]j=o'^j2"' with / = [log{t)\ 

and 6j E {0, 1} such that P = nj=o,<5j=i-fi'^^ and R^'' is obtained by computing a sequence 

R, R^^ , R'^^ , . . . , R'^' with / matrix-matrix multiplications. Iterated squaring to compute P works 
for semirings in general. In case of an idempotent semiring, we can use that approach for Q as 

well. It is straightforward to verify that (R + I)* = J2k=o^'' ^^^^ of idempotent semiring. 
We briefly recall the argument for this known result. Obviously, the result is true for t = 0. For 
t > 0, we first use the induction hypothesis and the idempotency of the semiring, in this way we 

have (R + I)* = (R + 1)*-^ ^ (R + I) = eI~=W ^ (R + I) zS\='^' + £1=0^' = ELqR'- 
Hence, for idempotent semirings, we can for instance compute X]a:=o(-'^['^i ^ ~'^2, ^1 A -'^2])'' and 
(M[$i A -'<I>27*I*i A -'$2])* with at most log{t) matrix-matrix multiplications and additions. 

With the presented algorithms, all formulas of CTL$ can be proved for the class of semirings 
that has been defined at the beginning of this section. The only missing step is the computation 
of the matrix N[<I>i A -''I>2) ^1 A -'<I>2] which has to be realized specificly for each semiring. In the 
examples below, we show that computation of this matrix can be done in most interesting semirings 
with an effort of O(n^) or below. If this is the case, then the effort for checking <I>i U^p <I>2 is in 
O(n^) whereas the effort for checking $1 U^^p ^2 for finite t is in 0{tn'^). In general, the effort 
grows linear in t and in the length of the formula and it grows at most cubic in the size of the 
automaton. 

Checking CTL$ formulas for more general semirings that are not order preserving requires some 
restrictions since otherwise an infinite summation may not be computable (for instance in case of 
divergent sums, non-existence of a fixpoint). Usually, matrix N[$i A ^^2,^1 A -■$2] cannot be 
computed for these semirings such that $1 U^p $2 can only be checked for finite t. Furthermore, 
the checking of <I>i AU^^p ^2 often cannot be done with the presented algorithm. If we restrict the 
formulas to those that do not contain $1 AU^p ^2 and contain $1 U^p $2 only for finite t, then 
the proposed algorithms can still be applied for modelchecking given that those parts are removed 
that terminate a loop due to ixi p. All decisions that rely on comparisons w txi p must be delayed 
to the end of the procedures since values can change in a non-monotonous manner. 

5 Bisimulation for Weighted Automata 

Bisimulation for weighted automata has been introduced in [S]. In [2], it has been shown that 
bisimulation is a congruence according to the operations of the process algebra GPA. Here, we 
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briefly rephrase the definition for bisimulation given in [HI El and prove afterwards that bisimilar 
states of an automaton are indistinguishable under CTL$ formulas. 

We consider only equivalence relations as bisimulations. Let TZ be an equivalence relation on 
5x5. S /TZ is the set of equivalence classes of 7^, C G S/TZ is an equivalence class of TZ and C[x\ 
is the equivalence class to which state x G 5 belongs. If we consider equivalence classes of different 
equivalence relations 7Zi, we use C-ji^ for an equivalence class from S jTZi. We define for C C 5: 
M(x,C)=E,6cM(x,y). 

Definition 5.1 An equivalence relation TZ for an automaton A is a bisimulation if and only if 
V(x, y) G 7^, VC7 G S/TZ and Va G C: 

1- E^ec^l^'"'^) = E^GC^(y'«'^)' equivalentlyMa{x,C) = Ma{y,C), 

2. a{x) = a{y), equivalently a(x) = a(y), 

3. f3{x) = f3{y), equivalently b(x) = b(y), and 

4- AP{x) = AP{y) where AP{x) is the set of atomic propositions satisfied by x. 

We define the union of two bisimulations TZi and TZ2 via the union of their equivalence classes. 
Thus TZq = TZi U 7^2 is characterized by the equivalence classes \x\ = C-ji^ [x] U C-jz,^ [x] for all 
X G 5. With this definition the union of bisimulation relations yields a bisimulation relation. 

Theorem 5.1 Let TZi and TZ2 be two bisimulations for automaton A, then TZ = TZi U TZ2 is also a 
bisimulation. 

Proof. The proof is a simple extension of the proof in one needs to consider the additional 
condition AP{x) = AP{y) of Def. 15. ![ which is however straightforward. Additionally, TZ is an 
equivalence relation since it results from the union of equivalence classes. □ 

Thus, the largest bisimulation for an automaton can be defined as the union of all bisimulations. 
We use the notation x ~ y for x, y G 5, if a bisimulation TZ with (x, y) G 7^ exists. The bisimulation 
can be extended to compare automata instead of states. This is commonly done for untimed 
automata as in ^26^ but requires slight extensions if applied to the general automata model presented 
here. Functions a and P require an additional condition. We define the union of automata in the 
usual sense and bisimulation of automata by means of a bisimulation relation on the union. 

Definition 5.2 Let Ai = {Si,ai,Ti, (3i) and A2 = i<S2,C(2,T2, P2) be two weighted automata de- 
fined over the same semiring K, identical alphabets C, and 5i fl ^2 = 0. The union Ai U A2 is 
defined as an automaton Aq = {SQ,ao,To, (3q) with 

• .So = 5i U 52, 



• ao{x) = ai(x) if X £ Si and 02 (x) for x G 52, and 

• (3o{x) = /9i(x) if X £ Si and (32{x) for x G 52- 

Automata Ai and A2 are bisimulation equivalent, if a bisimulation relation TZ exists for Aq and 




Ti{x,a,y) ifx,yGSi 
T2{x,a,y) ifx,yeS2 



otherwise. 



for all C G S/Ti: 



'xeCr\S\ 



'xecn52 




'xecn52 
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In terms of matrices, = -^i U A2 yields 



f 




Figure 2: Possible bisimilar model of the driving test example 



Example 5.2 Driving test, continued. Fig.\^shows a model which is bisimilar to the one in Fig.^ 
provided a semiring is given and functions T, a and fi and sets AP are appropriately defined. 
Let AP = {ok, learn} and AP{x) = {learn}\/x G {A, B,C, D, E, F,G,G' , L, L' , ABC, DBF} and 
AP{x) = {ok} for all x G {H,H'}. So by definition of a, (3, and AP, we have 3 candidates 
for equivalence classes {H,H'}, {G,G' , L, L'} and S\{G,G' ,L,L' ,H,H'} to fulfill conditions 2-4 
of the definition. By assuming T{x,a,y) 7^ iD for all arcs in Figs. Q and\^ and (D otherwise, we 
need to partition S\{L,L' ,H,H'} into sets {A,B,C, ABC} and {D,E,F,DEF} and to partition 
{L, L' ,G,G'} into {L,L'} and {G,G'}. For the Boolean semiring, addition is V in Def. \5.1\. 
condition 1, so it is straightforward to verify that this partition gives a bisimulation, i.e., L ~ 
L', G ^ G',H H',Ar^ ABC, B ~ ABC, C ~ ABC, D ~ DEF, E - DEF, and F ~ DEF. If we 
choose the semiring (R>o,+, -,0, 1), we achieve the same bisimulation if we define Mq for example 
as follows: 
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The fractions give the arc weights, while the index indicates the associated label, e.g., 1S/Lq{L,A) = 2; 
indicates To{L,l,A) = 1/2, which corresponds to a transition in the first automaton. Matrix entries 
that are D are omitted for clarity. 

The following theorem introduces the relation between bisimulation equivalence for weighted au- 
tomata and CTL$ formulas, which is similar to the relation between bisimulation and CTL in 
untimed automata. 

Theorem 5.3 If x ^ y, then 

1. x\= y\=^ for all $ which are logical combinations of atomic propositions, 

2. x^ [a]Mp.$ <^ y N Wxp-^, 

3. X ^ $1 f/^p $2 y N ^1 ^2 and 

where $1 and $2 o-re CTL$ formulas. 

Proof. 1. holds since AP(x) = AP{y) for x ~ y such that also all logical combinations of atomic 
propositions yield identical results. 

2. is proved inductively by assuming that for x ^ y: x \= ^ y \= ^. Then x \= [a][xip-*& 

y \= [a][x]p.^ since M(i(x,C) = Ma(y,C). Initially we know that AP{z) is the same for all z € C 
such that the relation holds for all $ which are logical combinations of atomic propositions. By 
induction the relation also holds for $ containing an arbitrary number of constructs of the form 
[a]i>3p<I>. For more general formulas we combine the induction used in this step with the induction 
presented for 3. and 4. below. 

3. and 4. have to be proved inductively over the number of occurrences of U^^p <I>2 and 
$1 AUl^ $2 in the formula and over the length t of the required paths. First assume for x ~ y: 

X 1= $1 ^y\=<^i and x |= $2 ^ 2/ |= 

which is proved for formulas $1 and $2 that do not contain $1 Ul^p ^>2 or $1 AUl^p $2- Now we 
prove X 1= $1 f/[^p $2 ^ y 1= ^1 ^>ap ^2 inductively over t. For t = we have: 

a(x)b(x) = a(?/)b(?/) 

such that the formula holds for x ~ y. Define for C € <S/ ~: ^(C) = h{z) for some (all) z and 
let (5(C, =1 if some (all) z G C: z \= ^ and ID otherwise. For t = 1, the following relation holds 
for X ~ y and all C e S/ 

a(x)E.ecM[$i A -$2, '^2]{x, z)b[^2]iz) 

a(x)M[$i A -$2, ^2]{x, C)^{C)6{C, $2) = 

a(y)M[$i A -$2, ^2](y, C)^(C)(5(C, $2) = 

a(y)E^gcM[$i A -$2,$2](y,2)b[$2](^) 

such that the required property is given for t = 1. Let b^(i) = E^g^M[$i A -'$2, *1'2](^, -2^)b['3>2](-2) 
for i e S. So, the aforegoing argumentation ensures that b^(x) = b^(y) if x ~ y. 
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For the induction step, we assume that the relation has been proved for t > 1 and we show that 
it holds for t + l. To simplify the notation, let P = M[$i A -.$2, ^1 A -.$2], Q = M[$i A -.$2, ^"2], 
and Kt = (e1=o(M[^i A -.$2, $1 A ^$2])'') M[$i A ^$2, $2]- We have to prove that 

Since a(x) = a(y) for x ~ y, we only need to show that 

Starting from the left side, we obtain by the induction assumption, 

g,gC7Rt(x,^)b[$2]W _ 

E.ecRt-i(x,^)b[$2](-^) + E.gc (P*Q) {x,z)h[^2]{z) = 
E,ec^t-iiy,z)h[^2](z) + E.ec (P*Q) {x,z)h[^2]{z) = 
j:,^c^t-iiy,z)h[^2]{z) + e^P'h'c 

At this point, wc a,rc done if Gx'P^ — P*. Obviously, x ~ y implies only P(x, C") = P{y, C) for 
all x,y (z C and all C,C' G S/ ^. However, we can show by induction that P(x,C) = P{y,C') = 
ipi{C, C) for all x,yeC and all C, C gS/ implies P*^(a;, C)P*^(y, C) = ^pk{C, C) for > 0. By 
definition, the statement is true for A; = 1. So for an inductive argument, we can assume that the 
result holds for A; — 1, then we have for an arbitrary C G <S/ ~ and all x, y G C: 

P\x,C') = EzesPix,z)P^-\z,C') 

Ec"65/~E.eC"P(^>^)P'-n-^,C") = Ec"65/~E.gC"P(^,^)V'fc-l(C",C") = 

Ec"^s/^Ezec"Piy,z)P''-Hz,c') = EzesPiy,z)'P'-Hz,c') 

So in summary, we obtain 

E,^cM^,^M^2]{z) 

g,gcRi-i(y>^)b[^2](^) +e,P*b'c, = 
E,ec^-i{y,z)h[^2]{z) + ej,P*b'c, = 
E.ecR^(y,^)b[$2](2) 

and the induction step is complete. This finishes considerations of $1 Ul^p $2- 

For $1 AU^^ $2, the above line of argumentation can be used completely analogously to prove 

a(a;)e^(Ello(M[^i A ^$2, ^1 A ^$2])'')M[$i A ^$2, -^1 A ^$2]e^ = iD 

a(y)e2/(Efc=o(M[$i A ^$2, ^1 A ^$2])'')M[$i A ^$2, A ^^>2]e'^ = ID 

and 

a(x)ej;(M[$i A -.$2, ^1 A -•$2])*e^ = ID a(y)ej,(M[$i A -i$2, ^1 A -i$2])* = ID 
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which proves x \= AU^jp $2 y \= ^1 AU^^p $2- We omit the details, since they provide no 
further insight. 

Finahy, to prove [/j^^ ^2 and $1 AUl;^p $2 for general and $2, we again use induction, 
namely over the number of occurrences of $1 U^^p ^2 or <I>i AU^p ^2 in a formula. Note that 
in the aforegoing argumentation, we did not use any other assumption for $1 and <I>2 than that 
X 1= $1 <^ y 1= and x |= $2 ^ y H ^2- Since this assumption holds here again by the 
induction assumption we can simply repeat the argumentation for $1 U^p ^2 and $1 AU^p ^2 
above for the induction step. □ 

The above theorem shows that one cannot distinguish between bisimilar states or automata by 
model checking CTL$ formulas. Thus, an automaton can be first reduced according to bisimulation 
equivalence to gain efficiency in subsequent model checking algorithms. For this purpose, first 
relation is computed, which can be done by a partition refinement algorithm, and then each 
equivalence class of ~ is substituted by a single state, which yields an aggregated automaton [S]. 
Afterwards, formulas are checked with the aggregated instead of the original automaton. In [5], it 
is shown that bisimulation is a congruence according to the composition operators of the process 
algebra GPA, which allows compositional analysis by interleaving reduction of components due 
to bisimulation equivalence and composition of components. In this way, a reduced automaton is 
generated to which model checking is applied. 

6 Examples of automata with specific semirings 

We present six examples in the following subsections. Two of the examples describe known types 
of automata which are presented in the proposed framework. In these cases we show that CTL$ 
model checking is related to logics presented specifically for these automata types. Afterwards we 
present new approaches for model checking. 

6.1 Untimed automata 

Untimed automata are defined over the semiring (B, V, A, 0, 1). For these automata a{x) = 1 for 
initial states and a{x) = for the remaining states. Similarly, /3(x) = 1 for terminating states and 
for the remaining states. T{x, a,y) = 1 describes the existence of an a-labeled transition between 
X and y. The Boolean semiring is ordered (0 < 1), the order is preserved by the operations and 
is the infimum. Therefore the conditions we proposed for model checking are observed. In the 
Boolean case all paths have the same weights, namely 1. 

For untimed automata CTL is a logic which is often used for model checking. We now show 
how the path formulas of CTL can be expressed by CTL$. State formulas defined via atomic 
propositions are obviously identical in both cases. 



CTL 



CTL$ 



^[$1 U $2] 
E[^i U $2] 



AF ^> 

EF $ 

AG $ 

EG $ 



EX<^ 
AX<!> 



^ {true U^Q ^$1) 
-^{true AU^Q ^$1) 



true UyQ ^ 
true C/>o ^ 
$1 AU^o $2 
$1 U^Q $2 
true AU^Q $ 
true U^n ^> 
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For the detailed description of the CTL-formulas see It is easy to show that for the 

Boolean case the model checking algorithms proposed above all have a finite runtime because for 

EU^^p <I>2i ^t^Mp and <I>i <I>2 are identical for all t > n. The reason for this behavior 
is that for an automaton with n states between two states a path of length < n or no path exists 
and since additionally all paths have the same weights and addition is idempotent, it is sufficient 
to consider paths up to length n if no longer paths have been defined explicitly via concatenation 
of [a] in the formulas. Consequently, the following relation holds for the Boolean semiring. 

- — ~ oo - — ~ n 

For the representation of CTL formulas using CTL$, paths of arbitrary length are considered. 
However, by considering paths of finite length and assuming that each transition of the automaton 
has a duration of one time unit, real time properties can be proved by CTL$ model checking. In 
this case CTL$ can be used to mimic formulas of the real time logic RTCTL |19j . 

Example 6.1 We consider the driving test example shown in Fig. Q over the Boolean semiring. 
In this case, each arc in the graph describes a transition with weight 1. Since for the Boolean case 
bisimilar automata cannot be distinguished by CTL$, model checking can be performed using the 
aggregated automaton shown in Fig. 

Since L' is the only initial state of the automaton, we have to prove whether a formula holds 
for V . Formula [true U^q ok) states that it is possible to pass the driving examination in an 
arbitrary number of steps. This formula is obviously satisfied by L' . The shortest path satisfying 
the formula start in V passes ABC and then enters H' . Thus, also the formula {true Ui,Q ok) is 
satisfied by L' for allt>l which means that the driving examination can be passed in 2 steps. The 
formula {true AU^^ ok) states that the examination is always passed. This formula is not satisfied 
by V because paths of infinite length exist which do not reach H' . Consequently, also formula 
{true AUt^Q ok) does not hold for V . 

In the Boolean semiring it is not possible to determine more detailed results about reaching state 
H' . We can only state that a path exists which reaches H' and that not all paths reach H' . CTL$ 
allows us to derive results about the length of the path reaching H' but not about the quantification 
of paths because the U operator equals EU in the Boolean semiring. This is different in the other 
semirings, we consider in the subsequent paragraphs. 

6.2 Probabilistic automata 

Probabilistic automata are defined over the semiring (R>o, + , •, 0, 1) with the additional restrictions 

EaG£ T.yes ^i^' a,y) = l for all x € 5 . 

A probability distribution is defined as the initial distribution and the sum of transition probabilities 
leaving a state is 1. These restrictions define a generative probabilistic model in the sense of [2H1 
because the automaton decides probabilistically which transition occurs next. Additionally, the 
automata model is similar to the model presented in ^21^ with additional possibility of labeling 
transitions. Probabilistic automata are ordered, the order is preserved by the operations and is 
the infimum which implies that model checking can be applied for this automata type. 

For probabilistic automata the logic PCTL has been proposed in pil. This logic contains, 
apart from state propositions and logical combinations of state propositions, the path quantifier 
$1 ^2 with a similar semantics as in CTL$. For finite t, the following relation between the 



18 



path formulas U and AU holds in probabilistic systems. 

The above relation does not necessarily hold for t = oo as shown in the example below. 

Now we consider CTL$ model checking for probabilistic automata. Interesting are the formulas 
$1 $2- For the remaining cases the algorithms presented in section |1] can be used because 
they require in these cases a finite number of steps. For the formulas with t = oo, matrix N[$i A 
-i$2; ^1 A -^^2\ has to be computed first which can be done as shown in the following theorem. 

Theorem 6.2 State x G S satisfies formula $1 ^2 if M[<I>i A -'^2,^2] is a substochastic 
matrix without a stochastic suhmatrix and 

a(2;)(E^o(M[$i A -«>2, $1 A ^^2]f)M.[^i A -«>2, ^'2]b[$2] = 

aL{x) (I - M[^>i A -.^>2, A ^^2]y^ M[^>i A -.$2, ^>2]b[^>2] ixi p 
Proof. The matrix representation of the formula has already been introduced. The relation 



^ (I - M[$i A ^$2, $1 A ^^2])^ = (I - M[$i A ^^>2, A ^$2]) 
fc=o 



-1 



is well known for absorbing Markov chains 24 under the conditions stated in the theorem. 



□ 



The theorem contains a method to decide for which states <I>i <I>2 holds. If M[<I>i A-i<I>2, ^1 A 
-■$2] contains a stochastic submatrix, then there exists a subset of states where $1 but not <&2 holds 
and this subset of states forms a trap according to the formula, i.e., the automaton can never leave 
the subset after entering it. Obviously <I>i C/^ $2 cannot be satisfied in these states and each 
path entering the subset does not count when path weights are summed. Thus, for subsets of 
states forming a stochastic submatrix, the rows in M[$i A -'$2;^i A -'^>2] might be set to to 
compute the result. After this modification the inverse matrix exists and the set of states satisfying 
$1 $2 can be computed in finitely many steps. 

Example 6.3 For the probabilistic case, we consider the driving test example with probabilities 
given in Examp. I5.ijl where also a bisimilar automaton with less states is presented. We can check 
the smaller aggregated automaton and consider the formula (true U^p ok) which is true if the test 
is passed with probability of at least p in an arbitrary number of steps. State L satisfies the formula 

a(L') iT[true,ok])'' ■ {h[ok]) > p 

k=0 

holds. Using the ordering of states (L' , ABC, DEF,G' , H') as given inTE". 
matrices and vectors. 

a = (1, 0, 0, 0, 0) , h[ok] = (0, 0, 0, 0, 1)"^ 



T[trne, ok] 



we obtain the following 
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(I - T[true,ok]y 



/ 4 3 2 1 1 \ 

2 3 2 1 1 

4 3 4 2 1 

4 3 2 2 1 

V 1 y 



and 



a (I - T[true, ok]y^ (b[ofc])' (1, 1, 1, 1, 1) 1 



which implies that the formula is observed for all p < 1. This means that after an arbitrary number 
of steps the driving test will be passed with probability 1. However, -iL' \= true AU^^ ok for all p. 
The example nicely shows that probability 1 does not mean that the result holds for all paths. This 
result is, of course, well known from probability theory. 



6.3 Max/plus automata 

Max/plus automata are defined over the completed semiring (R>oU{— oo, oo}, max, +, — oo, 0) with 
the computation a + — oo = max(a, — oo) = a and = a + = a. The weights of a path in 
max/plus correspond to the sum of weights of each transition on the path because multiplication 
is represented by the usual addition. If we consider several paths, then the maximum operator 
computes the weights of the path with the highest weights. Max/plus automata can be applied for 
various analysis purposes including the analysis of real time systems or communications networks 
and became very popular in the recent years. The max/plus semiring is ordered according to the 
usual ordering a <b <^ max(a, b) = b. Furthermore, the order is preserved by the operations and 
iD, in this case — cxd is the infimum of the semiring. In the definition of the transition function T, 
— oo is used to denote that an arc does not exist, which is the common usage of element iD. We can 
directly apply our model checking approach. 

Computation of the matrix N[$i A -''I>2, 'I'l A ^^2] requires the analysis of cycles in the matrix 
M[^i A -■<I>2, ^1 A -^^2]- xi, . . . ,XK is a. cycle if M[^i A -■$2, ^1 A -^^2]{xk,Xk+i) / (D (1 < /c < K) 

and xi = xk- The cycle has a positive weight if nfc=i M[<I>i A ^^2,^1 A ^^2]{xk, x^+i) > 0. It 
is well known that all cycles can be generated by composing minimal cycles and minimal cycles 
can be computed using some standard algorithms from graph theory. Element N[<I>i A -i$2)^i A 
^^2]ix,y) = 00, if a minimal cycle with a positive weight that contains arc {x,y) exists. The 
remaining elements in matrix N[$i A -i$2i^i A -'^*2]) which are not 00 can be computed from 
EI=o{M[^i a -$2, ^1 A -$2])^ 

Example 6.4 For this semiring, we need a different selection of T{x, a, y) to achieve bisimilar 
automata in Figs. ^and\^ since the maximal values of outgoing arcs of bisimilar states leading to 
the same class of states need to be equal. We select the following values which may be interpreted as 
distances the student has to drive or a quantification of the amount of stress he/she has to suffer. 
Matrix elements that are fl) and transition labels are omitted for clarity. 
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For instance, A ~ ABC since T{A,d,B) = 3 = T {ABC, d, ABC), max{T{A,e,D),T{A,e,E)) = 
9 = T{ABC,e,DEF), and T{A,f,H) = 3 = T{ABC,f,H') and further conditions of Def. EH 
with respect to a, (3 and AP hold as well. 

In this semiring, CTL$ considers the most costly (or stressful) ways to a driver's license exist. 
E.g. one can compute by the algorithm given in Sec. ^ that L \= trueU^2iok holds due to path vr 
through states L,A,E,G,L,A,B,H with ce(7r) = 21. For model checking we can use the bisimilar 
automaton given above, which contains less states and less arcs. So we check L' \= trueC/>2ioA; 
which holds due to path vr' through states L' , ABC, EFG,G' , L' , ABC, ABC, H . Both paths are of 
same length and have the same weights. 

6.4 Min/plus automata 

The min/plus approach is very similar to the max/plus approach. It is applied if one is interested in 
minimal weights instead of maximal weights. It is defined on the semiring (R>oU{cxd}, min, +, oo, 0) 
with an inverse order, i.e. addition becomes minimum x + y = min{x,y), multiplication becomes 
addition x^y = x+y, and x, y are ordered x > y iff x = min{x, y). The semiring preserves the order 
and D = oo is the infimum. Working with an inverse order is formally correct but rather contrary 
to intuition. Note that the inverse order of min/plus is the reason to use the notion of infimum 
and supremum rather than minimum and maximum in this paper. This avoids reformulation of 
the algorithms. Model checking algorithms can be applied analogously as for max/plus automata. 

Computation of the matrix N[<&i A -i$2;^i A -'^>2] is easier for min/plus than for max/plus. 
The reason is that minimal weights count. Since a cycle cannot reduce the weight of a path we 
have, as in the Boolean semiring 

— - oo -■ — - n 

N^*!"! A ^^>2,$i A ^^>2l = V (M[<I>i A ^$2,^1 A ^$2!)'' = V (M[$i A ^^>2, $1 A ^$2!)'' 

z — /fc=o ^ — 'fc=0 

6.5 Max/min automata 

The semiring (R>o U {00}, max, min, 0, 00) is useful to identify paths with respect to bottlenecks, 
since the weight of a path gives the minimum value observed through all of its arcs. We consider a 
communication network as an example. A weighted automaton models the network by using nodes 
for hubs and arcs for links between hubs. Each hub shows a certain utilization and each link has 
a bandwidth as a non-negative real number assigned to it. If there is no link between two nodes, 
we assume an arc with weight 0. In order to establish a point to point communication, we are 
interested in the existence of a connection between two nodes Xgtart and Xgnd that has a minimum 
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bandwidth uses less than A intermediate nodes and the employed nodes should have a utilization 
less than 7 to avoid saturated or overloaded nodes. 

To express this in CTL$, we first define atomic propositions and $2 as follows. A node 
X 1= <I>i if and only if its utilization is less than 7. A node a; ^ $2 if and only if it is node Xend- The 
following formula describes the property we are interested in $ = <I>i U^p $2 with p = fx, t = X. 
Model checking the automata by the algorithm for $1 U^^p ^2 given in Section |3 provides us with 
information whether Xstarts \= ^ or not. Note that the semiring is ordered, the operations preserve 
the order and (D = is the infimum. 

Computation of the matrix N[<l>i A ^^2, ^1 A ^^2] is easy in the max/min semiring because the 
weight of a path is determined by minimum weight of an arc on this path such that cycles cannot 
increase the weight of a path and N[<I>i A ^^2, ^1 A ^^2] can be computed by finite summation like 
for the Boolean semiring or the min/plus semiring. 

6.6 The expectation semiring 

In this subsection, we consider a semiring which is more complex than the previous and outline how 
our modelchecking approach can be extended to analyze also this system. However, the extension 
requires some additional steps. The proposed semiring is motivated by a semiring given in jl8j and 
allows the simultaneous computation of path probabilities and expected values of a set of paths. A 
value in the expectation semiring consists of two components {p, v) with p,v ^ R>o- The operations 
are defined as 

{Pi,^!)^ iP2,V2) = (pi ■P2,vi +V2) and {pi,vi) + (^2,^2) = (pi +P2,{pi ■V1+P2 ■V2)/{pi +P2)) 

where 0/0 = 0. We have iD = (0,0) and I = (1,0). Furthermore, we define the ordering > 
with (pi,fi) > {JP21V2) if Pi > P2 and vi < V2- Observe that this defines only a partial order 
since elements exist where neither {pi,vi) > (^21^2) nor {pi,vi) < {p2,V2) holds. The semiring is 
commutative because multiplication is commutative, it is not idempotent, and it is also not order 
preserving. 

Assume that we have an automaton over the expectation semiring where all transitions are 
labeled with a single label which will be suppressed in the sequel. As in probabilistic automata, 
let the sum of the first components pi of the weights {pi, Vi) of transitions i = 1,2,... that leave a 
state be smaller or equal to 1. Thus, the first values form a probability distribution of choosing a 
successor state, the second components might be interpreted as the costs of a transition. Assume 
that a{s) = I = (1,0) for one state, the initial state. Assume further that a predicate $2 for 
one state s' with /3(s') = I. Formula true U^^p^-^^2 holds if the probability of reaching the final 
state from the initial state in at most t steps is at least p and the expected costs are smaller or 
equal v. Similarly $1 U^p $2 holds if only nodes are touched where $1 holds on the way from 
the initial state to the final state, the remaining conditions are as in the previous case. For finite 
t the formulas can be checked with the proposed algorithms after some modifications that change 
all parts which are based on the order preserving property of the semiring. 

For this specific semiring, even results for t = 00 may be checked but this requires some tools 
from the analysis of Markov processes which are beyond the scope of this paper. 

The introduction of this rather unconventional semiring shows that the proposed method mod- 
elchecking approach can be extended to a very large class of models by using sophisticated semirings. 
However, if these semirings do not fall into the basic class defined at the beginning of section |^ 
model checking algorithms have to be adjusted specifically to their properties. 
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7 Conclusions 



We present a general approach for model checking weighted automata which covers classical types of 
automata like untimed or probabilistic automata as well as new types like max/plus and min/plus 
automata. The key idea is that transitions weights can be taken of an arbitrary ordered semiring 
which is an algebraic structure of very modest requirements. We present a modal logic CTL$ for 
this class of models that is build on top of CTL and allows to specify paths with respect to their 
length and weights. This yields a generic approach where new, different semirings automatically 
profit from algorithms and results derived for the general case, e.g. we present a bisimulation for 
CTL$ that is subsequently used to modelcheck an example under various weight assignments of 
different semirings. So far we presented analysis algorithms based on graphs assuming an explicit 
representation of states. This was for clarity and to limit the scope of the paper. Clearly, large state 
spaces are better treated by a symbolic representation. In the special case of the boolean semir- 
ing, binary decision diagrams (BDDs) and corresponding algorithms '14 are sufficient. However 
the general case requires the treatment of numerical values, such that corresponding extensions of 
BDDs like multi-terminal BDDs as in ^3] for instance are more appropriate. Furthermore, com- 
positional representations as in and compositional model checking are interesting candidates for 
modelchecking CTL$. 

Advantages of the approach presented here are foreseen for building analysis tools and to allow 
for model checking in different application areas like realtime scheduling and logistic networks. 
The latter is in the focus of a large DFG-funded collaborative research centre (SFB 559), with 
significant interest in modelchecking. The development of tools profits from our approach since it 
nicely matches an object oriented design, where model checkers of specific semirings can inherit 
functionality from an implementation of the general case. We currently work to integrate this 
approach into an existing CTL modelchecker within the APNN toolbox 

References 

[1] R. Alur and D. L. Dill. A theory of timed automata. Theoretical Computer Science, 126:183- 
235, 1994. 

[2] F. Baccelli, G. Cohen, G. Olsder, and J. Quadrat. Synchronization and Linearity. John Wiley 
and Sons, 1992. 

[3] F. Baccelli, B. Gaujal, and D. Simon. Analysis of preemptive periodic real time systems using 
the (max,plus) algebra. Research Report 3778, INRIA, 1999. 

[4] F. Baccelh and D. Hong. TCP is (max/-F) hnear. In Proc. SIGCOM 2000. ACM, 2000. 

[5] F. Bause, P. Buchholz, and P. Kemper. A toolbox for functional and quantitative analysis 
of DEDS. In R. Pujanger, N. N. Savino, and B. Serra, editors. Quantitative Evaluation of 
Computing and Communication Systems, pages 356-359. Springer LNCS 1469, 1998. 

[6] D. Beauquier and A. Slissenko. Polytime model checking for timed probabilistic computation 
tree logic. Acta Informatica, 35:645-664, 1998. 

[7] J. Bryans, H. Bowman, and J. Derrick. Model checking stochastic automata. ACM Transac- 
tions on Computational Logic, to appear. 

[8] P. Buchholz. Bisimulation for automata with transition costs, submitted for publication, 2000. 

[9] P. Buchholz and P. Kemper. Quantifying the dynamic behavior of process algebras. In L. de Al- 
faro and S. Gilmore, editors. Process Algebras and Probabilistic Methods, LNCS 2165, pages 
184-199. Springer, 2001. 



23 



[10] p. Buchholz and P. Kemper. Weak bisimulation for (max/+) automata and related models, 
submitted for publication, 2003. 

[11] J. R. Burch, E. M. Clarke, K. L. McMillan, D. L. Dill, and L. J. Hwang. Symbolic model 
checking: 10^° states and beyond. Information and Computation, 98(2):142-170, 1992. 

[12] E. M. Clarke, E. A. Emerson, and A. P. Sistla. Automatic verification of finite state concurrent 
systems using temporal logic specifications. ACM Transactions and Programming Languages 

and Systems, 8(2):244-263, 1986. 

[13] E. M. Clarke and J. M. Wing et al. Formal methods: State of the art and future directions. 

ACM Computing Surveys, 28(4):626-643, 1996. 

[14] E. M. Clarke, O. Grumberg, and D. A. Peled. Model Checking. MIT Press, 1999. 

[15] E. M. Clarke and R. Kurshan. Computer-aided verification. IEEE Spectrum, 33(6):61-67, 
1996. 

[16] R. Cleaveland, J. Parrow, and B. Stcffcn. The concurrency workbench: a semantics based tool 
for the verification of concurrent systems. ACM Transactions on Programming Languages and 
Systems, 15(l):36-72, 1993. 

[17] S. Eilenberg. Automata, Languages and Machines. Academic Press, 1974. 

[18] J. Eisner. Expectation semirings: flexible EM for learning finite-state transducers. In Proc. 
ESSLLI Workshop on Finite-State Methods in NLP, 2001. 

[19] E. A. Emerson, A. Mok, A. P. Sistla, and J. Srinivasan. Quantitative temporal reasoning. Real 
Time Systems, 4:331-352, 1992. 

[20] S. Gaubert. Performance evaluation of (max/+) automata. IEEE Transactions on Automatic 
Control, 40(12) :2014-2025, 1995. 

[21] H. Hansson and B. Jonsson. A logic for reasoning about time and reliability. Formal Aspects 
of Computing, 6:512-535, 1994. 

[22] M. C. Hennessy and R. Milner. Algebraic laws for non-determinism and concurrency. J. ACM, 
32:137-161, 1985. 

[23] Z. Jiang, B. Litow, and O. de Vel. Similarity enrichment in image compression through 
weighted finite automata. In D. Z. Du et al., editor, COCOON 00, pages 447-456. Springer 
LNCS 1858, 2000. 

[24] J. G. Kemeny and J. L. Snell. Finite Markov Chains. Springer, 1976. 

[25] W. Kuich and A. Salomaa. Semirings, Automata, Languages. ETACS Monographs on Theo- 
retical Computer Science. Springer, 1986. 

[26] R. Milner. Communication and concurrency. Prentice Hall, 1989. 

[27] M. Mohri, F. Pereira, and M. Riley. Weighted automata in text and speech processing. In 
A. Kornai, editor, Proc. of the ECAI 96, 1996. 

[28] R. van Glabbek, S. Smolka, B. Steffen, and C. Tofts. Reactive, generative and stratified models 
for probabilistic processes. In Proc. LICS'90, pages 130-141, 1990. 



24 



